1. Field of the Invention
This invention relates to an image correlator and to an image processing apparatus, such as an image processing circuit, a device for detecting the movement of a video signal, an autofocus camera or the like. More particularly the invention relates to an image correlator capable of detecting and following the movement of a specific portion of a moving image, to an image processing apparatus for detecting the degree of correlation of a predetermined still image with respect to a separately provided still image and determining the difference between the two still images, and to a signal adder which is preferably used in such an apparatus.
2. Description of the Related Art
An image correlator or an image comparator has the following function. That is, an image and a comparative image, serving as reference, are stored in a plurality of memory devices. The values of the luminance, hue and saturation, in the case of a color image, and the values of the luminance, in the case of a monochromatic image of corresponding points of the two images are compared with each other. The degree of similarity of the two images is determined by obtaining the degree of correlation, for example, within the range of "0", indicating a difference in the luminance level of at least 20 dB, to "1" indicating the difference equal to substantially 0 dB. Thus, image correlation can be obtained by determining the degree of similarity of the two images.
When detecting the degree of correlation between still images, the correlation can be obtained somehow even if the number of calculation processes is large because of allowance in time. On the other hands when determining and following the movement of a specific image by an image correlator, correlation is compared between the shape recognition of a first image and the shape recognition of a comparative image obtained after a predetermined time period (for example, for one frame). If correlation is high, the two images are determined as being substantially the same image. If correlation is low, the two images are determined as being entirely different images. For example, if the first image and the comparative image are shifted from each other in a lateral direction in a square display region, the amount of shift where a maximum correlation value is obtained corresponds to the amount of movement in the lateral direction, when obtaining correlation by shifting the comparative image in the lateral direction. Similarly, if the first image and the comparative image are shifted from each other in a longitudinal direction in a square display region, the amount of shift where a maximum correlation value is obtained corresponds to the amount of movement in the longitudinal direction, when obtaining correlation by shifting the comparative image in the longitudinal direction.
In order to measure the amount of shift, it is necessary to measure the amount of correlation by comparing the entire images by sequentially shifting the images from each other by the amount of a pixel and to obtain the amount of shift (measured, e.g., by number of pixels) where the amount of correlation has a maximum value. Hence, a storage capacity for a very large number of image signals, a calculation circuit for shifting the entire images, and a plurality of calculation circuits for obtaining correlation are required. It is difficult for the current technical level to satisfy the above-described requirements for ordinary moving images, although the need is strong.
When obtaining correlation between images, a matching technique for a specific image and a comparative image, serving as reference, is, in some cases, required. A technique for finding a point of an image where the local pattern of the density of the image is stepwise, and to find a point where a given density pattern is present is termed the matching technique. When matching a pattern with an image, a simple pattern to be matched, such as a step, a slope, a line or a point, or a "template" representing a known object is used. For example, a target template is fitted to an image obtained from a monitoring system, a target template is fitted to an image obtained from a navigation system, or a template having the shape of a star is fitted to an image of the sky. As an actual example, a pattern comprises a portion of an image, and the pattern is matched with another portion of the image. In this case, if two images photographed from different visual points are used and two portions of the two images are identified to represent the same place of a scene, stereoscopic parallax can be measured. It is thereby possible to determine the height and the distance of an object within a scene. It is also possible to measure the relative movement of an object within a scene using two images photographed at different times. That is, by executing the matching technique, the movement of a moving image can be followed by performing a tracking technique, such as raster tracking, a multidirection tracking technique or the like, in a longitudinal or lateral direction. However, it is difficult to achieve the above-described image matching technique, tracking technique or the like in real time unless parallel pipeline calculation or large-capacity high-speed calculation can be performed.
More specifically, a description will be provided illustrating the detection of hand movement. The above-described pattern matching technique is often used in movement detection according to image recognition. The movement is detected by comparing a photographed image with the image of the immediately preceding frame. For example, an image comprising about 8.times.8 pixels is matched with the preceding image. If it is assumed that matching is obtained by moving the image by two pixels in the x direction and three pixels in the y direction, this amount corresponds to the amount of hand movement. The amount is expressed by: EQU e(.xi., .eta.)=.SIGMA..vertline.g0(x-.xi., y-.eta.)-g1(x, y).vertline..
The value (.xi., .eta.) for providing the minimum value of the value e(.xi., .eta.) corresponds to the motion vector. Some recent microprocessors, for example, .mu.PD78335, include correlation calculation commands for facilitating the above-described calculation. However, even such microprocessors cannot deal with an image comprising a large number of pixels in real time. Particularly, if a minimum point is obtained by sequentially shifting the image by a point, the amount of calculation is very large, and therefore it is still difficult to obtain correlation in real time.
As described above, it is extremely difficult to obtain image correlation, and particularly difficult to detect the movement of a moving image in a real time.
It is difficult to calculate and output the amount of shift of pixels for providing the maximum correlation value between two images using an apparatus having a small circuit scale and capable of performing high-precision correlation calculation.
Next an adder used in an image correlator will be described.
Conventionally, an operational amplifier having a low-input-impedance common terminal where a signal from each input terminal is input via a respective resistor is generally used for a circuit for adding analog signals. A fixed voltage is applied to another input terminal of the operational amplifier. The gain of the operational amplifier can have a constant value according to the value of the resistor connected to each input terminal. Accordingly, predetermined weighting can be provided for an input voltage supplied to each input terminal.
However, the addition circuit using the operational amplifier requires a large area, and it is difficult to form such a circuit on a substrate for dealing with both digital and analog signals, because it is also necessary to provide a space for connecting analog circuits to digital circuits in addition to the area for the operational amplifier. Accordingly, there is no adder which satisfies requirements for reduction of circuit scale, reliability in circuit connection, and the like.